Machines such as, for example, excavators, loaders, dozers, motor graders, haul trucks, and other types of heavy equipment are used to perform a variety of tasks. During the performance of these tasks, the machines may operate in situations that are hazardous to an operator, under extreme environmental conditions uncomfortable for the operator, or at work locations remote from civilization. Because of these factors, the completion of some tasks by an onboard operator can be dangerous, expensive, labor intensive, time consuming, and inefficient.
One solution to this problem may include remotely controlling the machines. Specifically, an offboard operator located remotely from the machine, if provided with a visual representation of the machine and the work environment, could control operation of the machine from a more suitable location. This strategy has been implemented in the past and generally included providing the visual representation of the machine and work environment by way of live video feed broadcast from the worksite to the operator. The operator then was able to provide, via a graphical user interface, operational instructions that were subsequently sent to the machine for control thereof.
Although this strategy of remotely controlling the machines may have been successful in some situations, its use was limited and costly. Specifically, the visual representation of the machine and environment was typically limited to the number of cameras mounted to the machine and the view angles provided by those cameras. To improve visibility or provide different view angles, additional cameras had to be installed on the machine. Because the number of cameras on the machine relates directly to cost and, because the harsh environment of the worksite reduced the component life of the cameras, the initial and operating costs of the system were significant. In addition, wireless video feed in real-time requires large bandwidth, thereby further increasing the operating cost of the system.
An attempt at addressing the problems of high system cost and large bandwidth is described in U.S. Pat. No. 6,739,078 (the '078 patent) issued to Morley et al. on May 25, 2004. Specifically, the '078 patent describes a system utilized to remotely control construction equipment such as a backhoe at an isolated location via a data network, in which a user provides movement instructions via a graphical user interface (GUI) at a user PC. The GUI displays a side view and a top view visual representation of the movable elements of the backhoe (e.g., a boom, a stick, and a bucket). The visual representation is generated in response to movements of the boom, stick, and bucket of the backhoe, which are measured onboard the backhoe and transmitted to the user PC via radio frequencies. In this manner, an operator may remotely control hydraulic actuators onboard the backhoe to move the boom, stick, and bucket and, at the same time, view the resulting motions at a distant location in a cost effective manner.
Although the system of the '078 patent may provide a lower cost, more robust way to remotely view and control motions of construction equipment, its use may still be limited. In particular, because the system of the '078 patent provides a visual representation of only the boom, stick, and bucket, the operator may be unable to properly control engagement of the backhoe with its surrounding environment. In particular, without a representation of the worksite or an excavation surface at the work site, it may be very difficult, if not impossible, to adequately engage the bucket with the excavation surface. In addition, with the minimal visual representation described above, the operator may be unable to remotely move or orient the backhoe itself, or perform other necessary machine tasks.
The system of the present disclosure is directed towards overcoming one or more of the problems as set forth above.